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As policymakers and school leaders seek new ways to measure 
and improve teacher effectiveness, it's important for journalists 
and others to understand what is known about the topic so far, and what 
remains unsettled or unknown. This research brief does not synthesize all 
the studies in this highly technical field. But it does aim to improve the 
accuracy and clarity of reporting by exploring what the research says about 
timely questions surrounding the complex topic of teacher effectiveness. 

The brief is organized around several prevailing questions about teacher 
effectiveness in K-12 education. For each question, we've reviewed some 
of the most-important research, identifying key findings and tension 
points (Citations in the text refer to a list of sources in the bibliography). At the end of each section, we present a bottom- 
line summary of the research. 



BY STEPHEN SAWCHUK 



Nearly all of the studies cited here rely on the use of student test scores as a proxy for learning, a research practice 
that remains hotly debated. A full discussion of the value of standardized testing lies outside the scope of this paper, 
but we begin from the same assumption as many scholars: that standardized tests measure important aspects of 
student learning, but not the full breadth and depth of what students should know and be able to do. 



The brief draws on a review of over 40 specific research studies or research syntheses, as well as interviews with 
scholars who have used primarily quantitative research methods to analyze the relationships between teachers, their 
attributes, and student achievement. 




Are teachers the most important factor 
affecting student achievement? 

This has become the default first sentence of many speeches 
and reports on teacher quality. Recently, it's become common to 
clarify that teachers are the most important "school-based" factor 
in learning — a critical qualification, given that factors external to 
schools exert more influence overall on student achievement than 
any factors inside the school. 

A famous 1966 study by James Coleman found that background 
characteristics such as race, parental achievement levels, and family 
income swamped most other factors studied as determinants of 
student test scores. Decades of research have confirmed this study's 
general findings, with a 1999 paper estimating that 60 percent 
of variation in student achievement was attributable to such 
background characteristics. 1 



Researchers have been unable to link a significant share of the 
variation in student achievement — as much as 25 percent — to 
any particular input. Of the remaining share, attributable to what 
happens within school, researchers have linked most of that 
variation to teachers. 

It is difficult to cite an exact figure on what percent of the variation 
in achievement observed is attributable to differences in teacher 
effectiveness. Three economists in 1998 estimated that at least 7.5 
percent of the variation in student achievement resulted directly 
from teacher quality and added that the actual number could be as 
high as 20 percent. 2 

Researchers have found that school-based factors, including 
teaching, are more influential in math than in reading. A 1 999 paper 
puts all in-school factors, including school-, teacher-, and class-level 
factors, at approximately 21 percent of the variation in 10th grade 
mathematics achievement. It further estimated that 8.5 percent was 
directly due to teacher effectiveness. 3 



'Goldhaberetal, 1999. 
2 Hanusheketal, 1998. 
3 Goldhaberetal., 1999. 



Some researchers warn that other important factors that 
potentially affect achievement — such as the effect of principals 



THE NATIONAL EDUCATION WRITERS ASSOCIATION ^ visit us online at www.EWA.org 



> 



EWA RESEARCH BRIEf\ 



RESEARCH BRIEF > What Studies Say About Teacher Effectiveness 



and other administrators, and the interaction of teachers with 
the curriculum — have not been as carefully studied as teacher 
quality. 4 

It can be said: 

Research has shown that the variation in student 
achievement is predominantly a product of 
individual and family background characteristics. 
Of the school factors that have been isolated for 
study, teachers are probably the most important 
determinants of how students will perform on 
standardized tests. 



Are value-added estimations 
reliable or stable? 

Value-added modeling measures individual students' 
performance on tests over time, using prior test scores to predict 
future outcomes. Statistical controls attempt to screen out factors 
such as race, family background, and the effect of peers, so as to 
attribute the remaining variation in student academic outcomes 
to schools and teachers. 

At the level of individual teachers, such estimates vary considerably, 
pointing to differences in teachers' levels of skill. 

Some scholars say that of the measures of teacher effectiveness 
studied so far, value-added appears to be among the most 
promising. In one study, for instance, researchers used value- 
added estimates of teachers to predict the student-achievement 
patterns of some 3,000 students in 78 classrooms, and then 
randomly assigned teachers to these classrooms. The value-added 
models, while not perfect, were significant predictors of actual 
outcomes. 5 

At the same time, researchers have discovered that a host of 
factors contribute to measurement error in these estimates. 
These problems include the nonrandom assignment of students 
and teachers to schools and classrooms 6 ; different effect sizes or 
results based on the statistical models used 7 ; differences in the 
tests that supply the underlying data 8 ; the seeming instability of 
estimates of particular teachers from year to year 9 ; and the fade- 
out of teacher effects. 10 

In general, the variance in year-to-year estimates of individual 
teachers' performance could indicate measurement error. Some 
of these problems, like the problem of tracking and instability in 
the estimates, seem to be ameliorated by using additional years 
of student data for each teacher, though researchers continue to 



4 See, for example, Rothstein, R., 2010. 
s Kane and Staiger2008. 

6 Rothstein, J. 2009. 

7 See, for instance, Kane and Staiger 2008. 
8 See, for instance, Paypay 201 1 . 
'Koedel 2007. 



10 Rothstein, J. 2010. 

" Koedel 2009; McCaffrey et al., 2009. 

"Jackson 2010. 

,3 Jackson and Bruegmann 2009. 
"Harris and Sass 2009. 
,5 Hanushek 1992. 



debate this issue." 

The implications of these problems, both for policy and for 
research, are difficult to parse, and policy experts continue 
to debate the use of value-added as a component of teacher 
evaluations and for other purposes. 

Finally, the research on teacher quality suggests that other school 
factors may affect how effective teachers appear to be in these 
types of calculations. One study found that up to a quarter of the 
estimate of an individual teacher's value-added score depended 
on whether teachers were a good "match" for a particular school. 
Its author postulated that such factors as whether the teacher's 
teaching philosophy meshed with the school's culture and the 
choice of curricula might contribute to this match effect.' 2 

Many teacher groups argue that value-added measures fail to 
take into account the considerable role of school and district 
leadership. Researchers are still investigating the role of principals 
as distinct from teachers, but it is difficult to disentangle the two. 

Teachers' peers may also influence their effectiveness. At least 
one study has found that a teacher appears to improve when 
surrounded by more-effective colleagues. 13 But a second paper 
looking at this question found no consistent evidence that 
teachers who hold National Board Certification, an independent 
honor that teachers go through a rigorous process to obtain, have 
an impact on the effectiveness of their peers. 14 

For a longer discussion of the issue of value-added measurement 
and its place in policy, see Harris 201 1 . 



It can be said: 

Value-added models appear to pick up some 
differences in teacher quality, but they can 
be influenced by a number of factors, such as 
the statistical controls selected. They may also 
be affected by the characteristics of schools 
and peers. The impact of unmeasured factors 
in schools, such as principals and choice of 
curriculum, is less clear. 



What are the differences in 
achievement between students who 
have effective or ineffective teachers 
for several years in a row? 



Scholars have expressed the variation among teachers in many 
ways. An early paper on value-added dating from 1992, for 
instance, suggested that teachers near the top of the performance 
curve in a district could get an additional year's worth of growth 
out of students compared with the poorest-performing teachers. 15 
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They have also found that the variation in teacher quality is 
primarily within schools rather than between them, meaning that 
almost all schools have both effective and ineffective teachers. 

Despite much rhetoric from advocacy groups, the question 
of whether it's possible to dramatically change outcomes for 
struggling students by assigning them to several effective 
teachers in a row remains an open one. 

One of the most influential studies to assert this dates from 1 996. 
Drawing from mathematics test data onTennessee students linked 
to teachers over a four-year period, the study separated teachers 
by estimated effectiveness into quintiles. The researchers then 
tracked those students' progression from 3rd through 5th grade. 

They found that students assigned to three years of effective 
teachers outscored comparable students with three ineffective 
teachers by up to 50 percent points. The authors said that the 
effects of having several excellent teachers in a row accumulated 
over time. 16 More recent findings, however, have called into 
question the assumption that teacher effects can simply be added 
together year after year. 

A number of researchers have highlighted the "fade out" or 
depreciation of teacher effects over time,' 7 a phenomenon that 
has appeared in a number of studies using value-added modeling. 
What this means is that impact of a teachers' instruction on this 
year's scores does not seem to persist when those same students 
move on to other grades. One 2008 study, for instance, found that 
teacher effects from one year had only half their impact in the 
next year. 18 

Second, as noted earlier, estimates of individual teachers' effects 
vary from year to year, and are more volatile for smaller cohorts 
of students or when fewer years of data are incorporated into the 
estimates. 




Do teacher characteristics such as 
academic achievement, years of 
experience, and certification affect 
student test scores? 



Most of these characteristics have been examined using large 
sets of data in which teachers are linked to student scores. Several 
of the characteristics do indeed bear a relationship to student 
achievement, but in general, scholars say their effects tend to be 
somewhat weak or inconsistent across studies. 

Thus, on average, such characteristics matter. But there are plenty 
of cases in which teachers with advanced degrees, extensive 
experience, or specialized credentials are not noticeably more 
effective than their peers, and there are likewise many effective 
teachers without such credentials. 

The basic problem can be summed up by the conclusion of a study 
dating from 2005. It found large differences among teachers in 
terms of their value-added impact on student achievement and 
went on to note that this variation was not readily explained by 
factors such as graduate degrees or experience after the first few 
years in the profession. 19 A 1999 paper, meanwhile, estimated 
that only 3 percent of the variation in student achievement could 
be traced back to measurable teacher characteristics, such as 
teachers' academic profiles and degrees. 20 

Of these measures, there is good evidence that teachers gain 
in effectiveness with additional years on the job. 21 In general, 
value-added analyses show early career experience pays off in 
effectiveness steadily through at least the fifth year. This effect 
appears to be more consistent for elementary and middle 
schools than for high schools. 22 In addition, the impact of 
experience appears to be stronger than that of most other teacher 
characteristics. 23 




Therefore, policies that pair the best teachers with underserved 
students would need to identify teachers who consistently 
produce strong gains and ensure that such gains compound over 
time, if those initiatives are to have a lasting effect on student 
scores. Such policies have not been tested at scale. 

It can be said: 

Some teachers produce stronger achievement gains 
among their students than others do. However, 
estimates of an individual teacher's effectiveness 
can vary from year to year, and the impact of an 
effective teacher seems to decrease with time. The 
cumulative effect on students' learning from having 
a succession of strong teachers is not clear. 



Reviews of the empirical research on credentials, in general, point 
to consensus that teachers' math content knowledge seems to 
improve students' test scores in that subject. 

One study found this connection at the 1st and 3rd grade levels 
using a specially constructed measure of pedagogical content 
knowledge. 24 A second study looking at Florida test data found 
links between content-focused professional-development credits 
in math and secondary math achievement. 25 And one study found 
slight boosts in achievement for middle and high school students 
taught by teachers with an undergraduate or graduate degree in 
mathematics. 26 

Information on other content areas is sparse, but one study found 
a link between teachers' holding a bachelor's degree in science 
and student achievement in that subject. 27 



6 Sanders and Rivers 1 996. 
'Rothstein, J. 2009. 
"Kane and Staiger 2008. 
'Rivkinetal., 2005. 
"Goldhaber 1999. 



21 See, e.g., Rockoff 2004; Nye 
et al., 2004; Clotfelter et al, 
2007b; Clotfelter et al, 2007c; 
Harris and Sass 2007. 

22 Harris and Sass 2007 



3 Rice 2010. 

"Hill et al, 2005. 

s Harris and Sass, 2007, 2010. 

6 Goldhaber and Brewer 1 999. 

'Goldhaberand Brewer 1997. 



Studies are mixed on the attainment of advanced degrees and 
elementary-level student achievement; some studies show 
positive correlations, others negative ones. On balance the link to 
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achievement is likely tenuous at best. 28 Several studies have found 
that entering teaching with a master's degree of any kind does 
not boost achievement, relative to not holding such a degree, 
nor does earning such degrees seem to improve outcomes at the 
elementary level. 29 

Licensure test scores seem to matter more for math than for other 
subjects. They consistently appear linked to improved student 
achievement in that subject, at both the elementary level and 
at the high school level for algebra and geometry. Findings are 
mixed for other subjects. 30 



It can be said: 

Teachers improve in effectiveness at least over 
their first few years on the job. Characteristics 
such as board certification, and content 
knowledge in math sometimes are linked 
with student achievement. Still, these factors 
don't explain much of the differences in 
teacher effectiveness overall. 



As for certification, one study found that students taught by 
teachers with any sort of certification outperformed those 
without certification or who were certified out of field. 3 ' Another 
found that those taught by teachers with standard certification 
outperformed uncertified teachers or those with nonstandard 
certifications. 32 

Much of the information on teacher certification also seems to 
find benefits primarily for math. At the high school level, teachers 
with subject-specific credentials in math tended to boost students' 
scores more than those teachers who were not certified in that 
subject. 33 

Scholars have noted that teachers may affect learning more in 
mathematics, which tends to be taught exclusively in school, than 
in reading. 34 

Scholars have spent much time analyzing the effects of National 
Board Certification, but these numerous studies have mixed 
findings. In a review of the literature, a National Research Council 
panel concluded that evidence supports the notion that students 
taught by national-board-certified teachers on average have 
higher scores than those not taught by such teachers. But it said 
the evidence doesn't support the idea that the process itself 
makes teachers better at their craft. 35 

Teachers who enter the profession with specific sets of cognitive 
and noncognitive skills also on average seem to be slightly more 
effective than those who do not. 36 

The policy implications of these findings are, again, hard to 
parse. Generally speaking, the policy question concerns whether 
investing in certain teacher characteristics, (by paying a premium 
for teachers who hold National Board Certification or a master's 
degree, for instance), are cost effective relative to other possible 
investments. 

For a longer summary of the research literature on credentials, see 
Goe 2007. 




28 Nye et al., 2004 finds a positive 
correlation; see Harris and Sass 
201 1 for a full accounting. 
"Harris and Sass, 2007, Clotfelder 
et al. 2007a, 2007b, 2007c. 
30 Clotfelter et al., 2007a, 2007b, 
2007c. 



31 Goldhaber and 
Brewer 1999. 

32 Darling Hammond et 
al.,2005. 
33 Clotfelteretal., 
2007a, 2007b, 2007c. 
34 Nyeetal., 2004. 
35 Hakel etal., 2008. 



36 Rockoffetal.2008. 

37 Podgursky and Springer 2006. 

38 Springer 2010. 
39 Glazerman and Seifullah 2010. 



Does merit pay for teachers produce 
better student achievement or 
retain more -effective teachers? 



Performance-pay policies have been tried at many different 
points in the last several decades. Most offer monetary bonuses 
to teachers who boost student scores, participate in professional 
development, or meet other criteria, but they do not change base 
pay. 

The literature on performance pay is vast, and a full review lies 
outside the scope of this paper. Scholars say that the research 
questions around performance pay are hard to answer in 
just one study, especially since the questions vary. Do the 
programs encourage teachers to work harder and make them 
more effective at raising scores? Do they serve as a recruitment 
incentive, attracting high-quality teachers, over time changing 
the composition of the teacher workforce? Until recently, most of 
the research has focused on only the first question. 

Conclusions culled from random-assignment experimental 
studies, the research "gold standard," are limited. One review 
found just nine studies that used a random-assignment or quasi- 
experimental method to determine whether bonus programs 
raised scores; some of the studies looked at performance pay in 
countries outside of the United States. Those studies, in general, 
showed positive effects, but may not be applicable to the U.S. 
school system. 37 

In 2010, researchers at the National Center on Performance 
Incentives at Vanderbilt University released the results of a three- 
year experimental study on merit pay in Nashville middle schools. 
Under the experiment, math teachers who increased student 
scores received a significant bonus of up to $15,000. The study 
found no effects on achievement outside of 5th grade. Its authors 
concluded that the program had done little to change teacher 
practices. 38 

Also in 2010, preliminary results from a random-assignment 
experiment in Chicago on the Teacher Advancement Program, 
which includes merit pay as well as other features such as modified 
professional development, found no effects on achievement or 
teacher- retention rates. 39 
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A handful of other quasi-experimental studies have been mounted 
to study school reform plans that contain a performance-pay 
element. Using a method to create "synthetic"comparison schools, 
one study found apparent benefits for students whose teachers 
participated in the Teacher Advancement Program, a finding that 
stands in contrast to the Chicago experiment. 40 

Teachers participating in the pay-for-performance component of 
Denver's comprehensive ProComp teacher-compensation plan 
also appeared to boost achievement under certain conditions. 
The program requires teachers to set achievement goals with 
their principals. Those teachers who wrote the highest-quality 
objectives were associated with higher student achievement in 
elementary, middle, and high school than teachers who wrote 
lower-quality objectives. 

However, a comparison of schools participating in the pilot 
program with those not participating found mixed effects from 
the program. 41 

Factors such as the size of the bonus, the number of teachers 
permitted to receive it, and the methodologies used to award the 
pay all seem likely to shape the effects of such programs. 

It can be said: 

In the United States, merit pay exclusively focused 
on rewarding teachers whose students produce 
gains has not been shown to improve student 
achievement, though some international studies 
show positive effects. Research has been mixed 
on comprehensive pay models that incorporate 
other elements, such as professional development. 
Scholars are still examining whether such programs 
might work over time by attracting more effective 
teachers. 



Do students in unionized states do better 
than students in states without unions? 

Many studies have attempted to address the impact of 
unionization or collective bargaining on student achievement. 
The question needs careful parsing. 

It is true that students now tend to do better in heavily unionized 
states, like Massachusetts, rather than in those without required 
bargaining, like Alabama and Mississippi. But this simple 
correlation provides no information about whether unionization 
causes these achievement patterns. As the general public isn't 



likely to be aware of the difference between correlation and 
causation, it behooves reporters to explain the difference when 
reporting on this topic. 

Some research has been conducted on the causation question, 
and, as one 2008 paper summarizing the relevant literature found, 
results appear to be mixed. The studies tended to use different 
models and methodologies, choices that impacted their findings, 
the paper found. For instance, "point in time" studies tended to 
find positive impacts of unionization on academic achievement, 
while those looking at student growth over time tended to show 
negative impacts. 42 See Burroughs 2008 for a longer discussion 
and bibliography. 

The most recent study purports to use a "natural experiment" to 
compare performanceon SAT examsfrom the period between 1993 
and 1999, during which New Mexico had mandatory bargaining, 
to the period between 1999 and 2003, when bargaining was 
permissible but no longer mandatory. It compared performance 
during those time periods to achievement patterns in other states, 
controlling for factors such as state racial composition, poverty 
rates, and crime rates. 43 In addition, the author attempted to 
account for the fact that the change in bargaining laws probably 
would not have had immediate effects. 

The study found that mandatory collective bargaining was 
correlated with an increase in SAT scores, but a lowering of 
graduation rates. 

Critics of the study noted that federal data show that before the 
shift in state law, not all local teachers had voted in favor of union 
representation, raising questions about what phenomena the 
results actually reflect. 

It can be said: 

Students tend to do well in some heavily 
unionized states, but it isn't possible to 
conclude that it is the presence or absence of 
unions that cause that achievement. 



Hudson 2010. 
Slotnik et al., 2004. 
Burroughs, 2008. 
Lindy 2011. 



About the Author: Stephen Sawchuk is an assistant 
editor at Education Week, an independent national news 
organization based in Bethesda, Md. He reports on teacher 
quality and the teaching profession. 
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